WO 2005/052838 



PCT/SG2004/000382 



A Method and Apparatus for Building a Multi-Discipline 
and Multi-Media Personal Medical Image Library 



Field of the Invention 

This invention relates to a method and apparatus for building a multi-discipline and 
multi-media personal medical image library and refers particularly, though not 
10 exclusively, to such a method and apparatus using medical images of different formats 
and sources. 

Reference to Related Application 

15 This invention is related to the earlier US patent application serial number 10/307,190 
filed 28 November 2002 entitled °A Method and Apparatus for Creating Medical 
Teaching Files From Image Archives", the contents of which are hereby incorporated 
by reference (the "earlier patent application"). 

20 Abbreviations and Acronyms 

Throughout this specification the following abbreviations and acronyms shall have the 
following meanings: 

25 ACR: American College of Radiology 

AVI: Audio Visual Interleaved 

A multi-media file format usually with the file extension .avi 

BMP: Bitmap 

An image file format usually with the file extension .bmp 
30 DICOM: Digital Imaging and Communications in Medicine 

A standard medical imaging format and protocol 

DS: Dataset 

A document containing text and images that is useful for medical 

imaging research 
35 GUI: Graphical User Interface 

ECG: ElectroCardioGram 
EEG: ElectroEncephaloGram 
EMG: ElectroMyoGram 
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GIF: 
HTTP: 

5 

ID: 

JPEG: 
10 MIRC: 

MIRIP: 

MOV: 
MP3: 
20 PACS: 
PMIL: 

25 PNG: 

RM: 

RSNA: 
30 RGB: 

RGBA: 

SGI: 

35 



Graphic Interchange Format 

An image file format usually with the file extension .gif 
Hyper Text Transfer Protocol 

An application-level protocol for distributed, collaborative, hypermedia 

information systems 

Identifier 

A string used to identify a patient or a study in DICOM protocol 
Joint Photographic Experts Group 

An image format usually with the file extension jpg or .jpeg 
Medical Imaging Resource Center 

A distributed medical imaging repository and standard defined by 
RSNA 

Medical Imaging Repository Interfacing with PACS 

The name of the system based on the method disclosed in the earlier 

patent application 

Movie 

A multi-media file format usually with the file extension .mov 

MPEG (Moving Picture Experts Group) (audio layer) 3 

A multimedia file format usually with the file extension .mp3 

Picture Archiving and Communication System 

The clinical image archive 

Personal Medical Image Library 

The name of the system utilized the method and apparatus of the 
present invention 
Portable Network Graphic 

An image file format usually with the file extension .png 
Real Media 

A multi-media file format usually with the file extension .rm 
Radiological Society of North America 
Red Green Blue 

An image file format usually with the file extension .rgb 
Red Green Blue Alpha 

An image file format usually with file extension .rgba 

Silicon Graphic Incorporation 

An image file format usually with file extension .sgi 
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TF: 


3 

Teaching File 




A document containing text and images that is useful for medical 




education 


TIFF: 


Tagged Image File Format 




An image file format usually with file extension .tif or .tiff 


UID: 


Unique Identifier 




A number that uniquely identifies an object in the DICOM standard 


XML: • 


Extensible Markup Language 




Defined by the World Wide Web Consortium (W3C). 



10 

Background to the Invention 

Medical (e.g. radiology, pathology, endoscopy, and so forth) images are important 
building blocks for clinical support, teaching and research. In the digital environment, 

15 medical images may be organized and cross-referenced in a powerful and interactive 
fashion. As more clinicians use digitized images, a personal medical image library will 
become a valuable tool for on-demand learning, research, and exchange of date. 
However, a solution does not yet exist to build such a library that can collect and store 
medical images of various disciplines and formats. Medical Imaging Resource Center 

20 (MIRC), which was developed in 2001, is a standardized platform for exchanging 
image data. 

Many clinicians from all kinds of disciplines have their own hardcopy medical image 
collections for archive, teaching and research. But there is no open standard for them 
25 to organize and share them. Recently, the Radiological Society of North America has 
defined standards for Medical Image Resource Center. MIRC has the potential to be 
a worldwide set of standards defining teaching file and research data sets in the same 
fashion as DICOM has become the de facto standard for PACS. 

30 There are no known systems for obtaining medical images and other information from 
various sources to compose radiological teaching files and research datasets in a 
personal file library able to be shared with other libraries using the MIRC protocol. 

Summary of the Invention 

35 

In accordance with a preferred aspect of the invention there is provided a method for 
retrieving medical images from various sources and in different formats, to enable the 
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10 



creation of teaching files and research datasets, for the building of a personal medical 
image library, the method comprising: 

(a) retrieving a plurality of medical images from various sources; 

(b) storing the plurality of medical images in a database; 

(c) generating a database record for the teaching files and research datasets; 

(d) generating the teaching files and research datasets file; 

(e) saving the teaching files and research datasets into the database; and 

(f) generating at least one index of the teaching files and research datasets. 

The method may further include a searching mechanism for searching the teaching 
files and research datasets. 



The medical images may be from at least one discipline selected from: radiology, 
nuclear medicine, dermatology, pathology, ophthalmology, cardiology, neurology, 
15 endoscopy, angiography, biomedicine, ECG, EEG. and EMG. 

Preferably, the method is in accordance with MIRC schema. 

The method may further include anonymizing patient sensitive information, the patient 
20 sensitive information being able to be revealed to a generator of the teaching files and 
research datasets. Preferably, the patient sensitive information is not revealed 
publicly. The anonymization process may include the replacing of each item of the 
sensitive information with an anonymization code. The anonymization code may 
include a prefix, a randomly generated number, and a type. The prefix may be a short 
25 string of characters representing the generator of the sensitive information; and the 
type may represent the nature of the sensitive information. 

A check may first be performed to determine if the item of sensitive information has 
previously been anonymized and the anonymization code previously generated; and. 
30 rf yes, retrieving and using the previously generated anonymization code. 

The sensitive information may include one or more of: patient's name, patient ID, other 
patient's names, other patient IDs, patient's birth name, patienfs address, patient's 
telephone numbers, patient's mother's birth name, region of residence, country of 
35 residence, military rank, branch of service, patient comments, additional patient 
history, referring physician's name, referring physician's address, referring physician's 
telephone numbers, and all other person names. 
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In step (c), ACR codes may be entered as a result of system prompts. The ACR 
codes may be used for the at least one index of the teaching files. 

5 Indexing may b* by at least one of: title, abstract,- keywords, author^ affiliations, 
contacts, patient information, radiological codes, image format, image compression 
status, image modality, anatomic location, and ACR codes. 

For internal searching, patient sensitive information may be revealed, and for external 
10 searching patient sensitive information may be anonymized. 

In accordance with a second aspect of the invention there is provided apparatus for 
retrieving medical images from various sources and in various formats for creating at 
least one teaching file and research dataset; the apparatus including a database, an 
15 image retrieval interface able to retrieve medical images from various sources and in 
different formats, an MIRC server, a server, and a graphic user interface for operation 
on a user's machine. 



20 



The database is preferably a relational database for storage of all required 
information, including: database tables; database indexes; database scripts; and 
pointers to the medical images, teaching files and research datasets. 



The server preferably serves requests received from a user via the graphic user 
interface on a user's machine; the graphic user interface being for providing access 
25 functions and file editing ftinctions. 

The image server may include at least one of: a two dimensional image loader, a three 
dimensional image loader, a multi-media loader, and a telemetry loader. 

30 The two-dimensional image loader is for retrieving two-dimensional still images, the 
three-dimensional image loader is for retrieving three-dimensional still images; the 
multi-media loader is for retrieving multi-media files; and the telemetry loader is for 
retrieving telemetry data. 



35 The graphic user interface may include a PMIL client as a user interface able to run in 
a web browser or as a stand alone application on a user's machine, and provides 
MIRC editing functions. 
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The server may include an MIRC storage for providing an MIRC file storage service 
for the database and for the user's machine. The MIRC server may further include an 
MIRC query to provide queries as defined by the MIRC scheme. 
5 .... 

The at least one teaching file may be in accordance with a Medical Imaging Resource 
Centre standard. 

In a final aspect of the invention there is provided a computer useable medium 
10 comprising a computer program code that is configured to cause a processor to 
execute one or more functions to perform the method outlined above. 

Brief Description of Drawings 

15 In order that the invention may be fully understood and readily put into practical effect 
there shall now be described by way of non-limitative example only preferred 
embodiments of the invention, the description being with reference to the 
accompanying illustrative drawings in which: 

Figure 1 is a system block diagram that illustrates the main components of the 
20 apparatus; 

Figure 2 is a detailed structural diagram that illustrates in more detail the components 
of the apparatus of Figure 1 ; 

Figure 3 is a system flowchart that illustrates the steps to create teaching files and 
research datasets from medical images and other information; 
25 Figure 4 is a flowchart for anonymizing sensitive information; 
Figure 5 is a flowchart for anonymizing a DICOM image; and 
Figure 6 is a flowchart of input ACR code. 

Detailed Description of Preferred Embodiments 

30 

This system serves as a personal medical image library that stores and organizes 
medical images and metadata in a database, it can also be used to exchange image 
data in the global MIRC community. 

35 As shown in Figure 1 the system includes a database 1 , an image retrieval interface 2, 
a MIRC server 3, a web server 4, and a GUI 5. 
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The database 1 is preferably a relational database. The database 1 stores alt 
information for the system, including database tables, database indexes, and 
database scripts; and also stores the pointers to the physical files of the system. 

5 The image retrieval interface 2 is able to retrieve images of various disciplines and 
formats. The MIRC server 3 provides MIRC compliant functions, including MIRC 
query and MIRC storage. 

The Web server 4 services the various requests of the GUI, and the GUI 5 provides 
10 functions to access the system. GUI 5 also provides MIRC file editing functions. 

A more detailed structure is illustrated in Figure 2. The images retrieval interface 2 
includes a two dimensional image loader 21 that is used to retrieve 2-dimensional still 
images; a three dimensional image loader 22 that is used to retrieve 3-dimensional 
15 still images; a multi-media loader 23 that is used to retrieve multi-media files; and a 
telemetry loader 24 that is used to retrieve telemetry data. 

The web server 4 includes a PMIL servlet 25 running in the web server 4 to serve 
requests from the GUI 5. 



20 



The GUI 5 includes a PMIL client 26 as the user interface. The PMIL client 26 can be 
run in a web browser or as a stand alone application. 

The MIRC server 3 includes a MIRC storage 27 that provides MIRC teaching file 
25 storage service; and an MIRC query 28 provides queries as defined by the MlRC 
schema. 

As shown in Figure 3, the method for retrieving medical images comprises the steps of 
retrieving the medical images (31) from various sources and in different formats. The 

30 images may be in any number of formats. The images are then stored in the 
database 1 (32) and the patient information is anonymized (33). The database record 
for the teaching file or research dataset is then generated (34) and an XML file 
generated (35). The XML file is then saved into the database 1 (36). Indexes of the 
teaching files and research datasets are generated (37) and a searching mechanism 

35 is provided (38). The searching mechanism may be any known, appropriate database 
searching engine or application. 
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Different departments of a hospital generate a large number of medical images for 
various diagnosis purposes. Most images are in digital format, and are stored in digital 
media. Some images are typical or atypical; they are particularly suitable teaching 
and research. 

The first step of retrieving medical images (31) involves reading medical images 
stored in digital media. Various way of reading images are used including, but not 
limited to: 



10 • Open a single file from disk 

• Open multiple files from disk 

• Copy image from clipboard 

• Capture image from computer screen 

• Drag a single file into the system 
15 • Drag multiple files into the system 

• Drag a single folder into the system 

• Drag multiple fotders into the system 

Different image formats may be supported including, but not limited 

20 

• 2-Dimensional Still images 

> AVW, HDR/IMG (Analyze format version 8.0 and 7.5) 

> BMP (Windows Bitmap format) 

> DICOM (Digital Imaging and Communications in Medicine) 

25 > GIF 

> JPEG 

> JPEG 2000 

> PNG 

> PNM 

30 > PPG 

> RGB 

> RGBA 

> SGI 

> TIFF 

35 

• 3-Dimensional Still Images 

> AVW, HDR/IMG (Analyze format version 8.0 and 7.5) 
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> Animated GIF 

> MIRA 

> Muti-sliced TIFF 



• Multi-Media audio and video: 

> MOV 

> AVI 

> MP3 
10 > RM 

• Telemetry 

> Waveform for ECG, EEG, EMG 



15 For all supported formats, the relevant application for dealing with the format is 
provided, and is fully integrated into the operating system. All images are kept in their 
original format once retrieved. For two-dimensional images, two additional JPEG 
images may be generated for ease of browsing using a web browser. These 
additional images may be of the same size as the thumbnail images. For other image 

20 formats, an additional thumbnail image may be generated. 

When images are loaded from the open image dialogue, all supported images can be 
previewed in the dialogue, before they are actually loaded. In this way it is possible to 
load images of interest 



25 



30 



In step 32. after the images are loaded, then can be viewed in the image window and 
selected to add into teaching files and research datasets, and further stored in the 
database (1). 

Since medical images are usually large in size, they need a large amount of disk 
space to store. The image storage may be spanned to multiple storage media. When 
the current storage media in use is approaching being filled, a storage media with 
sufficient free space is located and subsequently used. 

35 Patient specific information retrieved from the clinical image archive is very sensitive 
and can only be referenced internally. It is not allowed to appear in teaching files and 
datasets, which may be published. Patient sensitive information can't be simply 
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removed from the teaching files. Sometimes it is required to be able refer back to the 
actual patient Therefore, patient sensitive information needs to be anonymized. 
There are two places where patient information may appear, in the database record 
and in DICOM files. 

5 

In the database record, the patient name and ID are stored. The process described 
below may be used to anonymize them. 

The anonymization process has the format of. 

10 

<Prefbo <Type> - <Number> 

Where, prefix is usually a small number of letters. The letters may represent the 
creator of the anonymized information. Typically, the letters may be the acronym of 
15 the creating institute such as, for example, "BIL". 

Type specifies the nature of the anonymized information. Since patient name and ID 
are most frequently referenced, a "P" and a "D" may be used to represent them 
respectively. An "X° may be used to represent all other types of information. Further 
20 classification is also possible, for example, an "A" can be used to represent address 
information. 

Number is a random number uniquely generated to distinguish the anonymized code 
from other codes. 

25 

For example, a code, "BILP-3388", represents a patient name, is created by an 
organization named "BIL°. 

As shown in Figure 4, the following process steps may be used to anonymize one item 
30 of patient sensitive information: 

41 . for each item of patient sensitive information 

42. check if this information is already anonymized by looking up the records of 
the database 

43. if yes, directly use the already anonymized code 

35 44. else, generate a new random number ensuring the new random number does 
not already exist (48) by checking the database (49) 
45. add prefix and type to the random number to form the anonymized code 
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46. replacing the sensitive information with the anonymized code 
47 keeping the correspondence relationship between the sensitive information 
and anonymized code securely in the database 

5 The system keeps the correspondence relationship between the sensitive information 
and the anonymized code in the database. It may also provide a method to reveal the 
information before anonymization by looking up the database, but this method is 
preferably only used internally. 

10 As shown in Figure 5, after a DICOM image is loaded and stored in the database, the 
system automatically will anonymize it by replacing all sensitive information in the 
DICOM fields (51). 

Sensitive information may include, but is not limited to: 

15 

• Patient* s name ^ 

• Patient ID 

• Other patient's names 

• Other patient IDs 
20 • Patients birth name 

• Patienf s address 

• Patient's telephone numbers 

• Patient's mother's birth name 

• Region of residence 
25 • Country of residence 

• Military rank 

• Branch of service 

• Patient comments 

• Referring physician'? name 
30 • Referring physician's address 

• Referring physician's telephone numbers 

• All other person names 

The following steps are then used: 
35 52. read the information embedded in the image 

53. for each information determine if it is patient sensitive information 
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54. anonymize the sensitive information using the process described above in 
relation to Figure 4 

55. repeat the process until there is no more information 

5 By using this anonymization method, no patient sensitive information is disclosed, but 
the generator can obtain the information. 

As shown in Figure 6, useful information such as, for example, patient name, ID, sex, 
age, race, and so forth, may need to be inserted into the database 1. Significant 
10 images may also be selected and inserted into the database 1. They may also be 
deleted and re-ordered. 

Author information and affiliation information may be retrieved automatically from the 
database 1 and then inserted into the teaching file database record. 

15 

A user interface 5 is provided to enter other necessary information such as copyright, 
information, title, difficulty level, access permission, publishing date, reviewer, 
abstract, keywords, clinical findings, image findings, radiological codes, diagnosis, 
diagnosis groups, pathology of condition, imaging of condition, differential diagnosis, 
20 similar cases, quiz and references, and so forth. 

ACR coding system is used in the Teaching File and Research Dataset record. An 
ACR code has the following format 

<aaaa> . <pppp> 

25 

Where <aaaa> is the anatomy part, and <pppp> is the pathology part They are digits 
from 0 to 9. 

The ACR code is input (61) by user remembering and inputting the radiological codes 
30 or by the system guiding the user to input the ACR code step-by-step, or digit by digit 
At each step, the system prompts the user with only the possible digits at the current 
position and the text of corresponding meanings. The first prompt (62) is for (he 
anatomy code. After the entry or selection of the anatomy code (63), a query is raised 
to determine if there are sub-anatomy codes (64). If not (65), the next section is the 
35 pathology section (66). If yes, a further prompt is raised (67) and the relevant data is 
entered or selected (68). This may be repeated until the pathology section is entered 
(69) whereupon the system prompts for the pathology code (70). Upon the relevant 
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pathological data being entered or selected (71) the system enquiries if there are more 
sub-pathology codes (72). If not, there are no more prompts (73) and the process 
ends (74). If yes, a prompt is raised (75) and the relevant data entered or selected 
(76). Steps 72, 75 and 76 may be repeated until other fields are selected (77), and 
5 the process ends (74). 

Based on the database record generated in step 34 of Figure 3, a teaching file or 
research dataset complying with the MIRC schema in XML format may be created. 
The teaching file or research dataset may also be previewed while editing, and it may 
1 o be reloaded for modification. 

A significant image inserted into the teaching files has two forms: a thumbnail, and a 
full image. The thumbnail image may be in JPEG format, and the full image may be in 
its original format or JPEG format 

15 

The case record and the XML file together with significant images (thumbnails and full 
images) may stored permanently in the database in step 36 for later access. Later 
access may be for indexing, searching and retrieving. 

20 The teaching files and research datasets stored in the database 1 may be indexed in 
various categories for searching purposes in step 37. The index categories may 
include title, abstract, keywords, authors, affiliations, contacts, patient information, 
radiological codes, image format, image compression status, image modality, 
anatomic location, and so forth. 

25 

Internet based searching mechanisms are provided in step 38. There may be two 
types of searching mechanisms: internal searching and external searching. For 
internal searching, patient sensitive information is exposed, whereas for external 
searching, this information is anonymized as is described above. 

30 

Computer useable medium comprising a computer program code that is configured to 
cause a processor to execute one or more functions to perform a method for retrieving 
medical images from various sources and in different formats, to enable the creation 
of teaching files and research datasets, for the building of. a personal medical image 
35 library, the method comprising: 

(a) retrieving a plurality of medical images from various sources; 

(b) storing the plurality of medical images in a database; 
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(c) generating a database record for the teaching files and research datasets; 

(d) generating the teaching files and research datasets file; 

(e) saving the teaching files and research datasets into the database; and 

(f) generating at least one index of the teaching files and research datasets. 

Whilst there has been described in the foregoing description preferred embodiments 
of the present invention, it will be understood by those skilled in the technology that 
many variations or modifications in details of one or more of: design, construction and 
operation, without departing from the present invention. 



